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Preface 


The  purpose  of  this  study  was  to  evaluate  the  performance  of  A-10 
aircraft  maintenance  units  and  aircraft.  The  study  was  also  a  vehicle 
to  demonstrate  the  applicability  of  Constrained  Facet  Analysis  to 
performance  evaluations  of  aircraft  maintenance  in  the  Air  Force. 

Results  showed  that  Constrained  Facet  Analysis  is  a  promising 
model  for  Air  Force  performance  evaluations.  Work  should  be  continued 
in  this  area  to  refine  the  use  of  Constrained  Facet  Analysis  for  Air 
Force  maintenance  management. 

Throughout  the  preparation  of  this  report  I  have  received  help 
from  a  number  of  people.  I  want  to  express  my  gratitude  to  Lt  Col 
Charles  T*.  Clark,  my  advisor,  for  his  infinite  patience  and  wisdom. 
Thanks  also  to  Lt  Col  Richard  L.  Clarke  for  help  in  the  correlations, 
and  to  Capt  Clinton  F.  Gatewood  for  his  help  and  advice  as  a  reader. 
Finally,  a  special  heartfelt  thanks  to  Larry  Stone  for  his 
encouragement  and  support  throughout  this  endeavor. 


Valerie  J.  Gonnerman 
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Abstract 


Performance  evaluation  is  required  to  obtain  important  feedback 
on  system  efficiency  for  management  decision  making.  For  Air  Force 
aircraft  maintenance  managers,  performance  evaluation  is  crucial 
for  determining  capability  and  evaluating  unit  efficiency  and 
effectiveness.  This  thesis  effort  applied  Constrained  Facet 
Analysis  (CFA),  a  linear  fractional  programming  technique,  to  the 
performance  evaluation  of  aircraft  maintenance  units  (AMUs)  and 
aircraft.  Empirical  data  for  three  AMDs  covering  a  five  month  period 
of  time  and  simulated  data  for  28  aircraft  was  evaluated  using  the  CFA 
and  Data  Envelopment  Analysis  (DEA)  computer  programs  at  the  University 
of  Texas.  Results  show  that  CFA  and  DEA  can  be  used  to  evaluate 
relative  efficiency  of  Air  Force  units. 


PERFORMANCE  EVALUATION  OF  A-10  AIRCRAFT 


MAINTENANCE  UNITS  AND  AIRCRAFT  USING 
CONSTRAINED  FACET  ANALYSIS 


I .  Introduction 


General  Issue 

Decision  asking  is  a  key  activity  and  o£ten  difficult  problem 
for  managers.  Good  decision  making  hinges  on  the  quality  and  appro- 
priateness  of  the  information  available  to  managers.  Performance 
evaluations  provide  information  inputs  into  the  decision  making 
process.  For  Air  Force  maintenance  managers,  performance  evaluation 
is  crucial  for  determining  capability  and  evaluating  unit  efficiency 
and  effectiveness.  This  research  considered  the  evaluation  of  Air 
Force  aircraft  maintenance  units  and  aircraft  using  the  Constrained 
Facet  Analysis  (CFA)  technique  developed  by  Lt  Col  Charles  T.  Clark 
(11). 

CFA  was  used  to  determine  relative  performance,  or  efficiency, 
of  A-10  maintenance  units  and  aircraft.  The  A-10  evaluation  was  based 
on  actual  data  for  aircraft  maintenance  unit  (AMU)  performance  and 
simulated  data  for  individual  A-10  aircraft.  The  relative  performance 
of  the  A-10  units  and  aircraft  studied  was  measured  in  terms  of  mission 
capable  time,  sorties  flown,  and  other  measures  related  to  availability, 


and  did  not  refer  to  inflight  performance  characteristics  such  as 
handling  and  maneuverability. 


This  research  was  useful  for  two  reasons:  it  provided  a 
performance  evaluation  of  A-10  aircraft  and  maintenance  units  for  use 
by  Air  Force  maintenance  managers,  and  it  contributed  to  the  body  of 
knowledge  concerning  CFA  and  its  applicability  to  management  decision 
making. 

Specific  Problem 

This  research  addressed  the  applicability  of  CFA  as  a  performance 
evaluation  technique  for  aircraft  maintenance  management.  Specifically, 
the  354  Tactical  Fighter  Wing  at  Myrtle  Beach  AFB  was  evaluated  based 
on  a  five  month  period  of  maintenance  data.  The  data  on  A-10  aircraft 
and  their  supporting  AMJs  were  used  to  identify  inefficiencies  and 
evaluate  possible  causes.  This  research  did  not  intend  to  correct 
deficiencies  but  to  direct  management  attention  to  areas  that  require 
further  study. 

Definition  of  Terms 

To  aid  the  reader's  understanding,  the  following  terms  are  defined 
as  they  were  used  in  this  thesis. 

1.  Aircraft  Maintenance  Unit  (AMU)  refers  to  the  maintenance 
organization  that  performs  direct  on-equipment  labor  on  assigned 
aircraft  and  that  is  responsible  for  the  management  of  all  maintenance 
actions  performed  on  the  designated  aircraft.  AMJs  are  found  in  the 
Tactical  Air  Forces  operating  in  accordance  with  MCR  66-5.  An  AMU  is 


paired  with,  and  supports,  a  specified  flying  squadron.  Generally, 
an  AMU  is  responsible  for  approximately  20  aircraft. 

2.  Decision  Making  Pnit  (DM7)  is  the  organizational  entity  being 
evaluated  by  Constrained  Facet  Analysis. 

3.  Effectiveness  is  a  measure  of  how  well  the  objectives  of 

a  system  are  met.  It  does  not  necessarily  refer  to  how  efficiently 
inputs  were  used  to  achieve  those  results  (7;  12). 

4.  Efficiency  is  similar  to  performance  for  purposes  of  this 
thesis.  Efficiency  refers  to  how  well  inputs  were  used  to  obtain 
outputs  (12). 

5.  Frontier  in  this  thesis  means  the  efficient  aircraft  or 
efficient  AMDs.  Inefficient  aircraft  or  AMDs  are  compared  to  a  subset 
of  efficient  aircraft  or  AMDs  on  the  efficiency  frontier.  The  frontier 
concept  is  similar  to  the  isoquant  concept  of  economics.  A  pictorial 
representation  of  a  frontier  is  given  in  Figure  1.  Note  that  a 
frontier  can  be  shown  graphically  in  two  dimensions,  but  a  graphical 
representation  of  the  frontier  is  impossible  to  show  for  multiple 
outputs  and  inputs  requiring  graphs  of  four  or  more  dimensions. 

6.  Performance  is  used  interchangeably  with  efficiency.  In 
this  thesis  performance  refers  to  A-10  availability  measures  of 
mission  capable  time,  flying  hours,  and  sorties  flown  as  they  relate 
to  resources  such  as  availability  of  parts  and  labor.  Performance  is 
the  measure  of  the  output  of  a  system  and  the  efficiency  of  the  process 
through  which  that  output  was  obtained  (12). 

7.  Production  Function  is  a  mathematical  concept  used  in  economics 
that  relates  output  to  inputs  in  an  equation.  For  a  production  function 


to  be  useful  for  performance  evaluation,  the  mathematical  relationship 
between  outputs  and  inputs  oust  be  specified.  A  general  form  for  a 
single  output  production  function  is  y  ■  .  .  .  xq),  which, 

stated  in  words,  means  that  output  y  is  a  function  of  the  vector  of 
multiple  inputs  (Xj,  .  .  .  xq)  (12). 

8.  Relative  Efficiency  Performance;  The  Constrained  Facet 
Analysis  model  empirically  measures  the  performance  of  a  given  unit 
relative  to  the  performance  of  other  units.  It  is  not  an  absolute 
standard.  For  this  thesis,  the  performance,  or  efficiency,  of  any  A-10 
aircraft  or  unit  is  measured  relative  to  the  other  A-10  aircraft  and 
units  in  the  study. 

Background 

Performance  evaluation  is  required  to  obtain  important  feedback  on 
system  efficiency  for  management  decision  making.  So  the  reader  will 
be  better  able  to  understand  the  need  for  and  importance  of  performance 
evaluation  in  management,  this  section  further  expands  the  definition 
of  performance.  Discussions  of  why  and  how  performance  is  evaluated 
are  also  included,  along  with  a  discussion  of  the  reasons  for 
understanding  this  research. 

The  A-10  Performance  System.  Performance  is  the  conversion  of 
inputs  into  outputs  by  a  specific  system.  This  conversion  can  be 
expressed  as  a  ratio  (17:323): 

Outputs  Achieved 

Performance  ■  - 

Inputs  Provided 

From  the  above  equation,  we  see  that  changes  in  any  factor  of  input  or 
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output  will  affect  the  overall  performance  of  the  system.  Performance 
could  be  improved  by  reducing  inputs  or  augmenting  outputs.  It  is  also 
affected  by  the  process  that  converts  inputs  into  outputs.  For  this 
thesis,  A-10  aircraft  and  AMDs,  and  the  inputs  and  outputs  identified 
for  each,  represent  performance  systems.  The  management,  mission, 
location,  and  time  period  are  environmental  factors  that  affect  the 
performance  of  the  systems.  Performance  evaluation  is  important 
because  it  provides  the  feedback  necessary  to  control  and  monitor  these 
systems.  The  A-10  systems  described  above  are  shown  pictorially  in 
Figure  2.  This  figure  serves  as  the  framework  for  the  rest  of  this 
thesis.  A  more  in-depth  discussion  of  how  this  system  is  adopted  to 
the  CFA  model  is  discussed  in  Chapter  II. 

Performance  Evaluation:  Why?  Performance  evaluation  provides 
important  feedback  for  the  managerial  activities  of  controlling, 
directing,  and  planning  system  operations  and  improvements. 

Performance  evaluation  should  be  a  factor  in  good  decision  making 
in  any  organization  and  is  applicable  to  all  managers,  military  as 
well  as  civilian.  It  helps  managers  make  better  decisions  by  giving 
them  information  concerning  the  function  and  efficiency  of  their 
organizations.  Therefore,  organizations  evaluate  performance  as  a 
basis  for  making  improvements  through  effective  decision  making.  The 
feedback  generated  by  performance  evaluation  is  an  invaluable  aid  in 
setting  realistic  production  goals  (17:323).  Performance  evaluation 
also  contributes  to  decisions  involving  resource  allocation.  Still 
another  use  for  performance  evaluation  involves  operational  planning. 

In  this  study,  planning  decisions  regarding  capability  and  mission 
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are  made  using  information  which  includes  performance.  Performance 
evaluation  is  an  essential  part  of  all  the  aforementioned  management 
situations  because  it  supports  and  enhances  decision  making. 

The  study  of  performance  applies  to  military  managers  for  three 
major  reasons.  First,  the  military  is  subject  to  close  scrutiny  by 
the  public  because  the  military  spends  a  large  portion  of  the  federal 
budget  and  must  justify  the  use  of  those  funds.  Military  managers 
must  operate  in  a  responsible  and  efficient  manner.  In  the  past 
year,  numerous  reports  revealed  the  exorbitant  cost  of  procurement  and 
upkeep  of  military  weapon  systems.  Reform  is  mandated  and  can  be  best 
achieved  through  performance  evaluations  which  bring  about  improvements 
in  efficiency  (13:12-30). 

Performance  evaluation  is  also  important  to  the  Air  Force  because 
of  the  impact  it  has  on  private  igdustry  through  defense  contracts. 
Maintaining  efficient  weapon  systems  can  be  achieved  by  the  military 
working  in  concert  with  the  defense  industrial  base.  Efficiency 
programs  in  both  the  military  and  the  defense  industry  will  improve 
weapon  system  reliability  and  combat  performance,  as  well  as  provide 
profit  and  growth  incentive  for  contractors  (15:50).  The  combined 
effect  of  these  two  forces  will  help  strengthen  the  economy  of  the 
United  States. 

Lastly,  the  Air  Force  must  balance  efficiency  with  effectiveness. 
The  Air  Force,  as  a  defense  service  for  the  United  States,  must  never 
be  forced  to  compromise  readiness  and  mission  accomplishment.  The 
successful  Air  Force  managers  will  be  those  who  understand  what 


-'A 


8 


performance  evaluation  is  and  how  to  use  it  in  monitoring,  controlling, 
and  improving  military  capability. 

Performance  Evaluation:  How?  There  are  many  performance 
evaluation  techniques  available  to  the  manager.  Selection  from  among 
the  various  techniques  depends  on  the  system  under  study  and  the  depth 
of  examination  desired.  The  techniques  that  will  be  discussed  in  this 
report  include  surveys,  ratios,  regression  analysis,  and  Constrained 
Facet  Analysis.  All  measures  of  performance,  their  differences  and 
their  applicability  will  be  discussed. 

Perhaps  the  least  quantitative  method  of  performance  evaluation  is 
the  survey.  This  method  can  give  the  manager  some  idea  of  how  workers 
and  managers  view  performance,  and  it  is  a  useful  starting  point  if 
differences  between  the  two  are  extreme  (15:58).  Surveys  can  be  brief 
in  scope  and  time  period,  or  more  complex.  For  example,  a  survey  could 
be  the  result  of  a  few  interviews  or  could  be  an  in-depth  survey  using 
written  questionnaires  with  a  statistical  analysis  of  results. 

Ratio  analysis  is  a  very  popular  and  common  form  of  performance 
measurement  that  managers  can  understand  and  use.  Air  Force  examples 
of  the  use  of  ratios  are  maintenance  manhours  per  flying  hour  and 
percent  fill  ratios  such  as  assigned/authorized  skill  levels.  The 
problem  with  ratio  measures  is  that  they  often  relate  only  one  input 
to  one  output  at  a  time.  When  the  number  of  inputs  and  outputs  of 
a  system  is  large,  the  number  of  ratios  that  can  be  derived  is 
staggering.  For  example,  if  there  are  4  outputs  and  10  inputs  of 
interest  to  management,  there  are  4  times  10,  or  40,  possible  output 
and  input  ratios  that  can  be  formed.  Even  experienced  managers  will 


have  difficulty  sorting  out  the  important  measures  and  using  them  for 
effective  decision  making.  It  is  difficult  for  humans  to  recognize  the 
multiple  interactions  among  a  large  number  of  ratios  using  cognitive 
skills  alone.  Even  ratios  which  relate  multiple  inputs  to  multiple 
outputs  are  difficult  to  interpret  because  the  relationships  between 
individual  inputs  and  outputs  may  not  be  apparent. 

Regression  analysis  is  another  commonly  used  performance 
measurement  technique  that  measures  several  inputs  against  one  output. 
Regression  analysis  determines  mathematical  relationships  between  the 
inputs  of  a  system  and  its  output.  In  this  way,  the  manager  can  see 
what  inputs  affect  the  output.  However,  even  multiple  regressions 
cannot  deal  with  situations  where  multiple  outputs  and  inputs  must  be 
considered  simultaneously.  Also,  since  regression  requires  a  priori 
specification  of  the  form  of  the  production  function,  this  method  is 
not  useful  for  evaluating  not-for-profit  organizations.  Production 
functions  for  not-for-profit  organizations  are  very  difficult  to 
determine  (6). 

Constrained  Facet  Analysis  (CFA)  overcomes  the  problems 
encountered  in  the  ratio  and  regression  analysis  methods  of  performance 
evaluation.  CFA  can  provide  one  performance  rating  for  multiple  input 
and  output  situations.  CFA  evaluates  relative  performance  of  systems 
under  study  by  building  a  frontier  of  efficiency  (7:4).  An  in-depth 
description  of  the  mechanics  of  CFA  is  included  in  Chapter  II. 

Evolution  of  Research.  This  thesis  grew  out  of  the  author's 
interest  in  aircraft  maintenance  and  in  performance  evaluation  as  a  way 
of  improving  maintenance  management.  It  is  this  author's  impression 


that  May  areas  of  maintenance  deserve  attention  and  need  improvement. 
It  ia  alao  evident  that  many  improvements  and  innovations  are  within 
the  grasp  of  wing  maintenance  managers.  Performance  evaluation 
contributes  to  improvement  because  it  identifies  efficiencies  and 
inefficiencies  and  provides  a  basis  from  which  to  manage. 

The  intention  in  this  thesis  was  not  to  study  a  particular  unit 
but  rather  to  show  one  way  of  evaluating  a  maintenance  organization. 
Further,  this  research  sought  to  show  that  the  method  of  evaluation, 
CFA,  can  give  relatively  quick  results  useful  at  all  maintenance 
management  levels. 

Discussion  of  this  thesis  with  Lt  Col  Clark,  AFIT/LSM  at  Wright- 

Patterson  AFB  OH,  led  to  the  selection  of  the  CFA  methodology  due  to 

its  appropriateness  for  performance  evaluation  of  multiple  input, 

% 

multiple  output  maintenance  units.  Detailed  reasons  for  the  selection 
of  CFA  are  discussed  in  Chapter  II. 

Research  Objectives 

1.  The  primary  research  objective  is  to  provide  a  logical  and 
useful  evaluation  of  A-10  maintenance  units  and  aircraft  using  the 
CFA  model. 

2.  The  author  also  hopes  to  show  that  aircraft  evaluation  can 
be  used  to  evaluate  unit  performance. 


3.  Another  major  objective  is  to  demonstrate  the  usefulness  of 
the  CFA  model  as  an  Air  Force  performance  evaluation  technique.  Use 
of  the  CFA  model  expands  the  manager's  repertoire  of  decision  making 


cools  by  providing  an  effective  means  of  obtaining  relative  measures 
of  performance,  productivity,  or  efficiency. 

4.  To  communicate  the  results  of  the  above  objectives,  it  is 
necessary  to  evaluate  CFA  data  and  translate  it  into  usable  management 
information.  Presenting  results  clearly  is  also  an  objective  of  this 
research. 

Research  Questions 

Several  research  questions  directed  this  research: 

1.  What  are  the  relative  efficiencies  of  AMDs  and  aircraft 
in  the  wing  studied? 

2.  Can  aircraft  performance  be  used  as  a  basis  for  unit  or 
wing  evaluation? 

3.  Did  CFA  provide  a  reasonable  and  adequate  evaluation  for 
this  thesis? 

4.  Could  the  CFA  results  be  communicated  to,  and  used  by, 

Air  Force  maintenance  managers? 


II .  Methodology 


Introduction 

This  chapter  discusses  the  Constrained  Facet  Analysis  (CFA)  model 
and  how  it  applies  to  performance  evaluation  of  A-10  AMUs  and  aircraft. 
The  development  of  CFA  and  its  usefulness  will  be  presented,  along  with 
the  particular  mechanics  of  the  model  that  apply  to  this  thesis.  Also 
included  is  a  discussion  of  the  data  collected  and  used  in  this 
research. 

Research  Model 

History.  Constrained  Facet  Analysis  was  developed  by  Lt  Col 
Clark  (11)  in  the  early  1980s  as  a  method  of  performance  evaluation 
and  decision  support.  CFA  is  a  direct  outgrowth  of  Data  Envelopment 
Analysis  (DEA)  which  was  developed  by  Charnes,  Cooper,  and  Rhodes  in  the 
late  19708  (10).  These  analysis  models  represent  breakthroughs  in  the 
performance  evaluation  field  because  they  are  the  first  to  deal  with 
multiple  outputs  as  well  as  multiple  inputs.  Further,  CFA  and  DEA  are 
especially  suited  to  nonprofit  organizations  because,  unlike  regression 
analysis,  they  do  not  require  a  priori  specification  of  mathematical 
production  functions  (6). 

CFA  and  DEA  have  been  tested  in  many  nonprofit  organizations. 


Bessent,  Bessent,  Kennington,  and  Reagan  performed  a  DEA  analysis  of  the 
Houston  Independent  School  System  in  1979  (6).  These  researchers  were 


able  Co  screen  schools  and  separate  Che  efficient  schools  from  Che 
inefficient  ones.  More  importantly,  they  vere  able  to  provide  managers 
with  empirically  based  information  for  goal  setting,  resource  alloca¬ 
tion,  and  operational  planning.  The  empirical  nature  of  the  data  and 
the  objectivity  of  the  results  are  features  of  DCA  and  CFA  that 
make  them  desirable  performance  evaluation  techniques. 

DEA  has  been  used  with  success  in  many  areas  of  research.  It  has 
been  used  to  evaluate  performance  in  hospitals,  military  recruitment, 
and  in  the  court  system  (4:28-32).  In  each  case,  DEA  provided  a 
performance  evaluation  of  the  organizations  that  identified  efficiencies 
and  inefficiencies.  There  appears  to  be  an  ever  increasing  range  of 
applicability  for  DEA  and  the  new  capabilities  of  CFA. 

Lt  Col  Clark  was  the  first  to  demonstrate  the  use  of  CFA  in 
evaluating  Air  Force  units  in  1983.  His  study  used  Air  Force  wings  as 
Decision  Making  Units  (DMUs)  and  determined  efficient  and  inefficient 
units  based  on  their  use  of  inputs  and  their  production  of  outputs  (11). 
Further,  the  research  cited  several  reasons  for  using  CFA  to  assess 
military  performance,  including  the  need  to  consider  multiple  inputs  and 
multiple  outputs  simultaneously,  the  need  to  have  access  to  information 
on  input  mixes,  and  the  difficulty  in  determining  and  quantifying 
production  functions  for  military  units  (1:166-167).  Clark's  study 
included  a  thorough  review  of  the  DEA  model,  and  the  theory  he  developed 
overcame  many  of  the  limitations  of  DEA  with  respect  to  rating 
efficiency  of  outlier  units  (11:171).  This  research  will  demonstrate 
the  use  of  the  CFA  model  in  another  military  context. 


Constrained  Facet  Analysis  (CFA)  Model.  CFA  is  a  linear  fractional 
program  that  calculates  a  relative  efficiency  for  each  unit  under  study 
(4:10).  The  objective  function  that  refers  to  unit  efficiency  in  the 
CFA  model  is  evaluated  subject  to  constraints  that  are  derived  from 
the  input  and  output  mixes  for  each  unit.  Those  interested  in  the 
formulation  of  the  CFA  program  should  review  the  works  of  Clark  and  the 
Bessents  (7;  8;  11).  CFA  is  an  extension  of  DEA  and  the  first  iteration 
of  CFA  is  equivalent  to  DEA.  CFA  differs  from  DEA  in  that  outlier  units 
are  subjected  to  an  iterative  analysis  procedure  which  expands  the  set 
of  frontier  units  used  as  a  standard  for  measuring  the  efficiency  of 
each  unit,  and  which  establishes  a  lower  bound  of  efficiency  for  these 
inefficient  outlier  units. 

Managers  can  use  these  models  without  understanding  the 
mathematical  forms;  however,  there  are  basic  concepts  that  must  be 
grasped  to  use  the  models  for  performance  evaluations.  First,  it  must 
be  stressed  again  that  CFA  determines  relative  efficiency  and  not  an 
absolute  standard.  The  efficiency  is  reported  on  a  scale  from  zero  to 
one.  Units  rated  one  are  efficient  relative  to  others  and  those  rated 
less  than  one  are  inefficient.  Use  of  these  measures  is  discussed  in 
detail  in  Chapter  III. 

Second,  managers  should  also  be  aware  that  CFA  provides  production 
rates,  substitution  rates,  and  values  if  efficient.  From  these  values 
managers  can  determine  what  changes  could  be  made  in  input  and  output 
quantities  to  improve  efficiency. 

Finally,  CFA  also  indicates  which  frontier  units  were  used  to 
establish  the  efficiency  rating  of  a  unit.  It  also  shows  which  frontier 


unit  most  closely  resembles  Che  inefficient  unit  in  input  and  output 
mix.  This  concept  and  others  will  be  clarified  in  Chapter  III. 


Summary  of  Why  CFA  Was  Selected.  The  CFA  model  is  eptly  suited  to 
the  analysis  of  A-10  AMU  and  aircraft  performance.  A  review  of  the 
reasons  why  CFA  is  applicable  to  the  research  includes: 

1.  The  model  deals  with  multiple  inputs  and  multiple  outputs 
simultaneously. 

2.  CFA  does  not  require  a  priori  specification  of  the 
production  function  and  is  therefore  appropriate  for 
evaluation  of  not-for-profit  organizations. 

3.  Empirical  data  is  used  in  the  analysis  of  performance 
using  CFA. 

4.  CFA  computes  the  relative  efficiency  of  the  units  under 
study  and  therefore  does  not  require  absolute  standards 
of  maximum  performance  which  usually  are  not  available ° 

5.  Further,  CFA  is  an  objective  mathematical  analysis  of 
performance. 

6.  The  CFA  computer  program  has  already  been  developed  and 
is  available  for  use  by  Air  Force  Institute  of  Technology 
(AFIT)  students  through  the  University  of  Texas  at  Austin. 

Se lection  of  the  Decision  Making  Unit  (DMU) 

Decision  making  unit  (DMU)  selection  is  a  very  important  step 
in  a  CFA  analysis.  DMUs  are  the  specific  organizational  units  to  be 
evaluated  and  identify  the  system  being  studied  (see  Figure  2).  The 
DMUs  for  this  analysis  were  A-10  AMUs  and  aircraft.  The  performance 
of  an  A-10  AMU  or  an  aircraft,  whether  efficient  or  inefficient,  was 


determined  by  the  CFA  program  in  terma  of  Che  level  of  output  an  AMU 
or  aircraft  achieved  with  its  levels  of  inputs.  DMU  selection  was  a 
critical  step  in  this  study  of  performance  because  it  influences  the 
selection  of  inputs  and  outputs. 

The  first  evaluation  was  done  with  AMUs  as  DMUs.  AMDs  were  used 
for  several  reasons.  In  general,  AMJs  are  homogeneous;  that  is,  they 
will  all  have  similar  inputs  and  outputs.  A  major  concern  was  the 
availability  of  data;  this  author  found  empirical  data  was  available 
for  the  evaluation  of  AMUs.  A  final  reason  for  selecting  AMU  as  the 
unit  of  evaluation  was  that  AMUs  are  closely  monitored  by  management 
and  are  already  subject  to  performance  evaluations  other  than  CFA. 

In  the  second  evaluation,  individual  aircraft  were  selected  as  DMUs 
for  several  reasons.  First,  the  choice  of  aircraft  provided  a  large  set 
of  observations,  which  is  consistent  with  the  requirements  of  CFA  and 
DEA  (4:13).  Also,  aircraft  represent  a  homogeneous  group  of  units. 

Thus,  we  can  assume  that  all  aircraft  draw  on  similar  resources  and 
produce  the  same  categories  of  outputs.  This  avoids  the  problem  of 
selecting  inputs  and  outputs  that  are  not  common  to  all  DMUs  and  insures 
comparability. 

From  a  conceptual  standpoint,  evaluations  of  aircraft  performance 
provide  useful  information  for  evaluating  AMU  performance  without  the 
complications  involved  in  studying  the  larger  units.  The  measures  of 
aircraft  performance  can  be  aggregated  to  obtain  measures  of  AMU  or  wing 
performance.  This  aggregation  provides  a  measure  of  wing  performance 
which  takes  into  account  aircraft  availability  and  the  wings  flying 


mission. 


Both  evaluations  of  AMDs  and  evaluations  of  aircraft  were 


accotaplished  to  show  the  flexibility  of  the  CFA  model.  Also,  these 
analyses  produced  results  that  Air  Force  aanagers  can  understand.  CFA 
gives  the  information  necessary  to  compute  ratio  measures  such  as  sortie 
rate  and  misf'on  capable  rate  which  are  familiar  to  managers.  For 
example,  ratio  measures  can  be  computed  for  an  inefficient  unit  from  the 
empirical  data  provided.  After  the  CFA  analysis,  ratios  representing 
efficient  operations  could  also  be  determined  for  the  same  inefficient 
unit  by  computing  the  ratios  from  the  values  if  efficient  which  are 
provided  by  the  model  (8:149).  This  gives  managers  targets  that  they 
can  understand  and  use  to  improve  efficiency,  targets  that  are  based 
on  the  unit's  present  condition  and  capabilities.  Examples  of  this  type 
of  conversion  of  data  will  be  important  in  the  analysis  of  results  and 
in  the  presentation  of  results  to  managers. 

The  selection  of  DMDs  was  done  prior  to  the  selection  of  input 
and  output  measures.  The  next  section  deals  with  the  selection  of  input 
and  output  measures  as  they  applied  to  the  DMDs  for  this  research. 

Input /Output  Measures 

General  Se lection  Criteria.  An  important  element  of  the  CFA 
method  is  the  selection  of  input  and  output  measures.  Although  the 
input  and  output  measures  for  AMU  and  aircraft  analysis  differ  somewhat, 
the  general  criteria  for  their  selection  are  the  same.  These  criteria 


1.  The  measures  should  be  complete  enough  to  measure  key 
goals  (outputs)  and  required  resources  (inputs)  of  the 
DMD  being  evaluated  (5:3). 

2.  The  relationship  between  inputs  and  outputs  is  also 
an  important  consideration.  Inputs  should  be  defined 
directionally;  that  is,  when  an  input  is  increased, 
an  increase  in  output  is  expected  (6:1360). 

3.  It  is  important  that  all  DMUs  use  some  of  each  input 
to  produce  their  outputs  (4:16). 

4.  Even  inputs  that  are  not  controllable  by  managers  should 
be  included  (5:16). 

The  remainder  of  this  section  describes  input  and  output  measures 
for  AMU  evaluation  followed  by  measures  for  aircraft  evaluation. 

AMO  Output  Measures. 

Output  1 :  Number  of  sorties  flown  per  time  period.  A  sortie 
includes  takeoff,  flight,  and  full  stop  landing  of  an  aircraft.  For 
this  thesis,  the  type  of  sortie  (i.e.,  training,  combat,  deployment) , 
was  not  differentiated;  thus,  all  sorties  were  added  together  to  yield 
a  single  sortie  measure  of  output.  In  the  AMU  analysis,  sorties  were 
accrued  on  a  monthly  basis  and  represented  all  sorties  on  all  AMD 
aircraft  for  a  specified  calendar  month.  Sorties  were  selected  as  an 
output  measure  because  in  a  fighter  wing  the  number  of  sorties  generated 
are  critical  in  assessing  effectiveness  and  performance.  Sorties 
represent  a  measurable  and  important  AMU  goal  and  were  therefore 
included  in  the  CFA  analysis. 
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Output  2:  Mission  capable  time  per  time  period.  Mission  capable 
tine  for  an  AMU  is  the  sun  of  all  the  mission  capable  time  on  each  AMU 
aircraft.  An  aircraft  is  mission  capable  when  it  is  available  for 
flight  or  otherwise  not  engaged  in  flying  but  requiring  no  maintenance 
or  supply  parts.  For  this  thesis,  mission  capable  includes  fully 
mission  capable  (FMC)  and  partially  mission  capable  (PMC).  The  measures 
of  mission  capable  time  included  in  this  research  are  monthly  totals.  A 
major  readiness  goal  of  AMUs  is  to  maximize  their  mission  capable  time, 
and  for  this  reason  mission  capable  time  is  used  as  an  output  measure 
for  the  analysis. 

AMU  Input  Measures. 

Input  1:  Number  of  aircraft  possessed  per  time  period.  Each  AMU 
has  specific  aircraft  assigned.  This  author  considered  the  number  of 
aircraft  each  AMU  had  available  for  use  to  be  an  input  measure.  The 
number  of  aircraft  possessed  represents  a  monetary  and  equipment 
resource.  Intuitively,  one  would  expect  the  number  of  sorties  and 
the  amount  of  mission  capable  time  to  increase  with  an  increase  in 
the  number  of  aircraft  available. 

Input  2:  Reciprocal  of  Not  Mission  Capable  Maintenance  (RNMCM). 

Not  mission  capable  maintenance  is  a  measure  of  the  time  aircraft  were 
unavailable  for  operations  because  maintenance  was  required  or  in 
progress  to  return  the  aircraft  to  mission  capable  status.  In  the 
source  data  for  this  thesis,  not  mission  capable  maintenance  time  was 
measured  in  hours  per  month  and  was  the  sum  of  all  not  mission  capable 
maintenance  time  on  all  AMU  aircraft.  The  reciprocal  of  NMCM  was  used 
as  input  data  for  the  CFA  model  to  preserve  directionality.  One  would 
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expect  outputs  to  increase  as  the  not  mission  capable  time  decreases  or 
its  reciprocal  increases.  The  reciprocal  of  NMCM  is  a  surrogate  measure 
of  maintenance  capability. 

Input  3;  Reciprocal  of  Not  Mission  Capable  Supply  (RNMCS).  Not 
mission  capable  supply  is  a  measure  of  the  time  an  aircraft  was 
unavailable  due  to  lack  of  supply  parts.  It  was  treated  in  the  same 
manner  as  NMCM  time.  The  reciprocal  of  NMCS  is  a  surrogate  measure  of 
the  supply  support  provided  to  an  AMU. 

Input  4;  Number  of  flying  days  per  time  period.  A  flying  day  is 
a  calendar  day  for  which  flying  was  scheduled  and  accomplished.  The 
number  of  flying  days  per  month  was  used  as  an  input  measure  for  the 
AMU  because  it  measures  the  availability  of  time  used  to  achieve 
outputs.  It  was  expected  that  as  the  number  of  flying  days  increased, 
sorties  and  mission  capable  time  would  also  increase,  thus  meeting  the 
directionality  criterion  for  inputs. 

Input  5:  Fix  rate  per  time  period.  The  fix  rate  is  the  percentage 
of  code  3  aircraft  malfunctions  in  a  month  which  are  repaired  within 
eight  hours  after  the  malfunction.  A  code  3  break  refers  to  an  aircraft 
malfunction  discovered  in  flight  which  prohibits  further  flight  of  the 
aircraft  until  the  malfunction  is  fixed.  For  this  research  the  fix  rate 
was  obtained  directly  from  source  data.  The  fix  rate  is  a  surrogate 
measure  of  the  quality,  quantity,  and  management  of  resources,  as  a 
high  fix  rate  indicates  a  good  supply  of  resources  or  that  resources 
are  being  managed  well.  A  low  fix  rate  could  indicate  shortfalls  in 
manpower,  training,  supply,  or  management  effectiveness. 


Aircraft  Output  Measures. 

Output  1 :  Number  of  sorties  flown  per  time  period.  A  sortie  was 
defined  in  AMO  output  1.  In  the  aircraft  analysis  sorties  represent  the 
total  weekly  sorties  per  aircraft.  Sorties  can  be  considered  outputs  of 
each  aircraft  and  were  therefore  included  in  the  CFA  analysis. 

Output  2;  Number  of  flying  hours  accrued  per  time  period.  Flying 
hours  include  the  time  an  aircraft  is  flying  and  performing  a  mission. 
Flying  hours  are  important  because  they  are  allotted  to  Air  Force  wings 
on  an  annual  basis.  This  thesis  treats  the  flying  hours  of  each 
aircraft  as  an  activity  which  contributes  to  the  overall  wing  flying 
hour  program.  Flying  hours  were  estimated  per  aircraft  per  week  for 
this  research. 

Output  3:  Number  of  mission  capable  hours  accrued  per  time  period. 
The  concept  of  mission  capable  was  defined  in  AMD  output  2.  Aircraft 
undergoing  maintenance  of  any  kind,  including  inspection  and  washes, 
or  needing  parts  for  flight  are  not  considered  mission  capable.  Main¬ 
taining  a  pool  of  mission  capable  aircraft  is  a  key  goal  of  a  wing.  For 
this  analysis,  mission  capable  was  also  considered  to  be  a  goal  of  indi¬ 
vidual  aircraft  and  was  estimated  on  a  weekly  basis  for  each  aircraft. 

Aircraft  Input  Measures. 

Input  1 :  Mean  Time  Before  Failure  (MTBF)  per  time  period.  The 
MTBF  is  a  measure  of  reliability  that  is  inherent  in  each  aircraft. 

One  would  expect  that  the  greater  the  MTBF,  the  greater  mission  capable 
time  would  be.  Because  the  aircraft  data  was  simulated,  the  MTBF  is 
an  estimate.  This  simulation  will  be  discussed  in  the  aircraft  data 


section. 


Input  2:  Reciprocal  of  Not  Mission  Capable  Maintenance  (RNMCM) 
per  time  period.  The  reciprocal  of  NMCM  vas  used  as  an  input  measure 
for  aircraft  for  the  same  reasons  it  vas  used  in  the  AMU  analysis. 

The  differences  in  using  this  measure  for  aircraft  vas  that  NMCM  vas 
estimated  for  each  aircraft  for  a  one  veek  time  period. 

Input  3:  Reciprocal  of  Not  Mission  Capable  Supply  (NMCS)  per  time 
period.  This  input  is  similar  to  that  used  in  the  AMU  analysis.  As  in 
input  2  ( 1/NMCM)  for  aircraft,  it  vas  estimated  for  each  aircraft  for 
one  veek. 

Input  4:  Number  of  sorties  scheduled  per  time  period.  The  nund>er 
of  sorties  scheduled  vas  used  as  an  input  because  of  its  effect  on 
the  number  of  sorties  an  aircraft  flev.  An  increase  in  the  number  of 
sorties  scheduled  for  an  aircraft  should  increase  the  number  of  sorties 
that  vill  be  flovn  by  that  aircraft.  The  ratio  of  sorties  flovn  to 
sorties  scheduled  shovs  hov  much  aircraft  are  falling  short  of  their 
sortie  commitments. 

Input  5:  Manhours  per  flying  hour  per  time  period.  Manhours  per 
flying  hour  is  a  measure  of  the  maintenance  time  required  to  produce 
each  flying  hour.  It  vas  used  as  a  surrogate  input  measure  of  manpover 
applied,  inherent  aircraft  reliability  and  maintainability.  It  vas 
expected  that  an  increase  in  this  input  would  relate  to  an  increase  in 
outputs.  In  this  analysis  manhours  per  flying  hour  ratios  were 
estimated  based  on  data  from  a  time  period  of  one  veek. 

Data 

This  section  addresses  the  data  used  in  the  CFA  analysis  for  this 
thesis.  Discussion  begins  with  AMU  data  followed  by  aircraft  data.  The 


section  concludes  vith  a  summary  of  data  collection  and  use  of  the  CFA 


program. 

AMU  Data.  The  source  data  for  the  AMU  analysis  vas  obtained  from 
the  354th  Tactical  Fighter  Wing  at  Myrtle  Beach  AFB,  Myrtle  Beach,  South 
Carolina.  The  source  documents  for  this  data  were  the  354th 'a  monthly 
maintenance  data  analysis  reports  (16).  Five  months  of  data,  from 
October  1983  to  February  1984,  were  used  to  compare  efficiency  for 
three  AMDs,  the  353rd,  355th,  and  356th. 

The  desired  input  and  output  measures  were  obtained  directly 
from  the  source  data  and  used  as  input  data  for  the  CFA  program.  The 
reciprocals  for  NMCM  and  NMCS  were  calculated  and  scaled  by  1  x  10^ 
for  ease  of  computation.  Table  I  shows  the  data  set  used  for  the 
CFA  evaluation  of  AMUs. 

Aircraft  Data.  The  data  set  for  aircraft  evaluation  was  simulated 
and  was  not  empirical.  The  simulation  was  based  on  the  empirical  data 
used  in  the  AMU  analysis;  however,  no  real  aircraft  were  represented. 

For  each  output  and  input  measure,  a  range  was  determined  based  on 
available  AMU  data.  This  range  was  scaled  to  estimate  individual 
aircraft  for  a  weekly  time  period.  Extreme  values  (see  aircraft  numbers 

10,  11,  27,  and  28)  were  included  in  the  data  set  to  represent  hangar 

1  .  2 
queens  and  high  flyers. 


A  hangar  queen  is  the  maintenance  term  used  to  denote  an  aircraft 
that  has  been  grounded  for  several  days  due  to  severe  maintenance  or 
supply  problems. 

2 

A  high  flyer  is  an  aircraft  that  performs  above  average  in  terms 
of  number  of  sorties  and  flying  time. 


Aircraft  Maintenance  Unit  Analysis  Data 
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An  example  of  how  aircraft  values  were  estimated  for  mission 
capable  hours  is  shown  below. 

1.  From  354TFW  data  the  nuotoer  of  mission  capable  hours  was 
divided  by  the  number  of  aircraft  for  each  month.  Each  of  these  numbers 
was  then  divided  by  four  to  give  the  average  number  of  mission  capable 
hours  per  aircraft  per  week: 


53639.4  Mission  Capable  Hours/Month 

October  ■  . — - 

80.1  Aircraft  x  4  Weeks/Month 

■  167.4  Mission  Capable  Hours/Aircraft/Week 


49149.3  Mission  Capable  Hours/Mbnth 

November  *  . ■—  ■  1  . . . - 

77.5  Aircraft  x  4  Weeks/Month 

■  158.5  Mission  Capable  Hours/Aircraft/Week 


52841.9  Mission  Capable  Hours/Month 

December  *  . . .  . . . . 

79.7  Aircraft  x  4  Weeks/Month 

“  165.8  Mission  Capable  Hours/Aircraft/Week 


50737.8  Mission  Capable  Hours/Month 

January  ■  ■  ■  ■  - - 

78.5  Aircraft  x  4  Weeks/Month 

■  161.6  Mission  Capable  Hours/Aircraft/Week 


44462.4  Mission  Capable  Hours/Month 

February  ■  - 

71.6  Aircraft  x  4  Weeks/Month 

■  155.2  Mission  Capable  Hours/Aircraft/Week 


2.  The  average  of  the  above  was  used  as  the  average  mission 


capable  hours  per  aircraft  per  week: 


167.4  +  158.5  +  165.8  +  161.6  +  155.2 


161.7 


3.  The  average  (161.7)  was  used  as  a  base  Co  create  mission 
capable  hours  for  28  aircraft  (see  Table  II).  For  aircraft  numbers  10 
and  11  very  low  mission  capable  hours  were  used  because  these  aircraft 
were  intended  to  represent  hangar  queens.  Aircraft  14  and  15  have  low 
mission  capable  hours  to  represent  aircraft  that  have  relatively  average 
inputs  but  still  perform  below  average  in  mission  capable  time  due  to 
poor  management,  inherent  reliability  of  the  aircraft,  or  other  problems. 

All  output  and  input  measures  were  created  in  a  similar  fashion: 
an  average  was  obtained  from  empirical  data  and  a  realistic  range  was 
created.  Extreme  values  were  used  to  indicate  special  cases  such  as 
hAgar  queens  or  high  flyers. 

Data  Collection  and  Computer  Use.  The  primary  source  of  data  for 
this  thesis  was  the  354  Tactical  Fighter  Wing  at  Myrtle  Beach  AFB.  The 
data  was  provided  by  a  wing  maintenance  officer  and  arrived  in  the  form 
of  monthly  data  analysis  reports  and  Quality  Assurance  reports  (16). 

This  author  found  that  the  typical  wing  data  analysis  reports  gave  many 
input  and  output  measures  required  for  CFA  analysis.  These  reports  also 
highlighted  the  need  for  an  evaluation  such  as  CFA  because  they  merely 
list  or  display  tables  of  measures.  They  do  not  relate  inputs  to 
performance  outputs  nor  do  they  attempt  to  compare  unit  performances  via 
single  measures  of  efficiency. 


Two  other  sources  of  data  were  sought  but  found  of  little  use  in 
this  study.  Air  Force  Logistics  Command  (AFLC)  supplied  several 
computer  tapes  of  discrepancies  on  A-10  aircraft.  This  data  could  not 


TABLE  II 


Aircraft  Analysis  Data 


AFCT 

OUTPUTS 

INPUTS 

No.  of 
Sorties 
Flown 

No.  of 
Flying 
Hours 

No.  of 
MC 

Hours 

MTBF 

* 

RNMCM 

** 

No.  of 
RNMCS  Sortie 8 

**  Scheduled 

MH/FH 

1 

18 

28.8 

167 

8 

50 

170 

18 

32 

2 

19 

30.4 

166 

8 

85 

200 

19 

33 

3 

20 

32.0 

165 

9 

65 

160 

20 

34 

4 

21 

33.6 

164 

9 

60 

170 

21 

32 

5 

22 

35.2 

163 

10 

45 

180 

22 

33 

6 

23 

26.8 

162 

10 

70 

250 

22 

34 

7 

24 

38.4 

161 

11 

80 

200 

23 

37 

8 

25 

40.0 

160 

11 

75 

190 

25 

36 

9 

26 

41.6 

159 

11 

90 

200 

25 

35 

10 

2 

2.0 

20 

6 

40 

70 

2 

120 

11 

5 

7.0  . 

40 

5 

35 

80 

5 

120 

12 

10 

16.0 

130 

6 

80 

150 

12 

50 

13 

11 

17.6 

131 

6 

70 

140 

12 

49  • 

14 

12 

17.6 

129 

5 

65 

130 

13 

48 

15 

13 

20.8 

128 

5 

60 

200 

13 

50 

16 

14 

20.8 

70 

8 

70 

160 

15 

49 

17 

15 

24.0 

80 

8 

80 

90 

15 

48 

18 

18 

27.2 

155 

10 

60 

90 

18 

36 

19 

19 

32.0 

154 

10 

65 

90 

19 

35 

20 

20 

30.0 

153 

11 

50 

150 

21 

34 

21 

21 

32.0 

152 

11 

55 

160 

21 

34 

22 

22 

30.0 

151 

12 

45 

200 

21 

35 

23 

23 

35.0 

150 

13 

40 

190 

22 

36 

24 

24 

36.2 

149 

14 

35 

180 

24 

37 

25 

25 

37.0 

148 

14 

30 

200 

25 

38 

26 

26 

38.5 

146 

14 

50 

140 

25 

37 

27 

30 

48.0 

150 

15 

60 

200 

30 

32 

28 

31 

49.6 

139 

16 

70 

250 

30 

38 

*  Mean  Time  Before  Failure  (operational  hours/number  of  failures) 
**  Scaled  by  100,000 

MH/FH  ■  Manhours  per  flying  hour 

RNMCM  *  Reciprocal  of  not  mission  capable  maintenance 
RNMCS  *  Reciprocal  of  not  mission  capable  supply 
AFCT  *  Aircraft 


be  used  because  input  and  output  measures  were  not  identified  or  related 
to  specific  time  periods.  Future  studies  using  CFA  may  find  some 
information  available  through  AFLC,  but  they  must  be  careful  to  obtain 
data  suitable  for  CFA.  Tactical  Air  Command  (TAC)  supplied  A-10  AMU  and 
wing  data  for  all  CONUS  A-10  bases.  This  data  arrived  too  late  for  this 
thesis  but  is  available  for  future  studies. 

Data  was  extracted  from  the  maintenance  reports  and  collated  for 
each  analysis  (see  Tables  I  and  II).  This  data  was  typed  directly  into 
a  data  file  at  the  University  of  Texas  at  Austin  and  used  in  their  CFA 
program.  The  specific  file  requirements  are  available  through  Lt  Col 
Clark  at  the  Air  Force  Institute  of  Technology  (AFIT)  or  from  the 
Educational  Productivity  Council,  Department  of  Educational 
Administration,  University  of  Texas  at  Austin  (8:6).  For  large  data 
sets,  researchers  are  advised  to  prepare  files  on  punch  cards  in  advance 
of  using  the  University's  CFA  program. 

CFA  and  DEA  programs  were  run  on  each  data  set,  AMU  and  aircraft, 
at  the  University  of  Texas  on  25  and  26  June  1984.  This  researcher 
found  the  CDC  Cyber  at  the  University  of  Texas  relatively  easy  to  access 
and  the  CFA  and  DEA  programs  easy  to  use,  giving  rapid  results. 

DEA  was  exercised  as  an  additional  analysis  because  it  was 
available  at  the  time  the  CFA  analysis  was  performed,  and  DEA  was  able 
to  use  the  same  data  sets.  The  DEA  program  provides  the  upper  bound 
efficiency  measures  of  CFA;  results  of  both  analyses  are  given  in 
Chapter  III.  The  major  intent  of  the  thesis  remained  the  same  —  to 
show  CFA  applicability  to  maintenance  management.  CFA  analysis  alone 
proved  to  be  sufficient  for  the  objectives  of  this  research. 


III.  Findings  and  Analysis 

Introduction 

This  chapter  includes  results  and  a  discussion  of  those  results 
for  AMD  and  aircraft  analyses.  The  chapter  is  divided  into  two  major 
sections,  one  on  AMDs  and  one  on  aircraft.  Each  section  includes 
findings  for  CFA  and  DEA  analyses.  Additionally,  the  AMD  section 
includes  a  correlation  analysis  of  input  and  output  measures. 

In  both  the  AMD  and  aircraft  analyses,  several  output  options  were 
analyzed  by  the  CFA  program.  Each  section  includes  computer  analyses 
of  each  data  set  using  all  outputs  and  an  analysis  using  each  output 
alone.  In  all  cases,  all  input  measures  were  included.  This  will  be 
discussed  in  greater  detail  in  each  section. 

Aircraft  Maintenance  Dnit  (AMD)  Analysis 

Phase  1 :  Efficiency  Ratings .  The  primary  result  of  the  CFA 

analysis  was  the  upper  and  lower  bound  efficiency  ratings  obtained  for 

3 

each  AMD  in  the  study  (see  Table  III).  Of  the  15  AMD  evaluations,  six 
were  rated  as  inefficient  relative  the  others.  Closer  examination 
showed  that  the  353rd  AMD  was  inefficient  three  times,  the  355th  AMD 
was  rated  inefficient  twice,  and  the  356th  AMD  was  rated  inefficient 


Table  III  also  contains  output  shortage  amounts  and  input  surplus 
amounts  which  will  be  used  later  in  this  chapter  in  computing  "values 
if  efficient." 


esults  for  AMU  Analysis  Using  Two  Outputs 


only  once.  The  apparent  efficiencies  and  inefficiencies  of  these  AMU  a 
are  summarized  in  Figure  3. 

The  relative  efficiencies  and  inefficiencies  that  resulted 
from  the  analysis  can  also  be  viewed  seasonally.  All  inefficiencies 
occurred  in  the  December  through  February  time  periods.  February 
appeared  to  be  a  particularly  “bad"  month  for  the  AMDs,  as  all  three 
were  rated  inefficient  and  received  the  lowest  of  the  inefficient 
ratings  in  this  month.  The  apparent  seasonal  effect  on  efficiency 
is  portrayed  in  Figure  4. 

Figure  4  may  also  be  used  to  stake  inferences  regarding  AMD 

stability  as  it  relates  to  the  performance  parameters  outlined  in  this 

research.  Care  must  be  taken  in  this  area  because  a  longer  time  span 

would  probably  give  a  better  representation  of  stability.  However, 

* 

from  the  efficiency  rating  observed  in  the  five  month  period  repre¬ 
sented,  one  notes  that  the  356th  AMD  appeared  to  be  the  most  stable 
performer.  The  353rd  AMD  also  appears  to  have  a  degree  of  stability, 
although  AMD's  performance  gradually  deteriorated.  In  contrast, 
the  355th  AMD  appeared  to  be  unstable  by  having  more  pronounced 
fluctuations  in  performance  ratings. 

Thus,  from  a  simple  comparison  of  efficiency  measures,  a  manager 
can  obtain  useful  feedback  on  unit  performance.  First,  a  manager  can 
look  at  the  efficiency  measures  and  see  what  units  are  performing 
poorly  relative  to  the  others.  Secondly,  the  efficiency  measures  can 
be  compared  to  time  changes,  in  this  case  months,  to  see  if  there  are 
seasonal  effects  on  performance.  Lastly,  the  efficiency  versus  time 


NUMBER  OF  SORTIES  (Output  1) 


OCT  83 


NOV  83 


DEC  83 


JAN  84 


FEB  84 


MONTHLY  TIME  PERIODS 


Figure  4.  Seasonal  Changes  in  Efficiency  of  354  TFW  AMUs 


analysis  can  bn  usad  Co  giva  cha  manager  a  rough  fan l  for  unit 
stability  and  how  units  compare  over  tins. 

Efficiancy/Iffactiaanass  Comparison.  Tha  use  of  the 
efficiency  ratings  can  be  taken  a  step  further  to  obtain  some  estimates 
of  unit  effectiveness.  This  can  be  attempted  for  output  measures  for 
which  standards  have  already  been  set.  In  this  thesis,  two  additional 
CFA  runs  were  made  using  each  output  individually.  The  efficiency 
measures  for  each  analysis  were  plotted  versus  their  respective 
outputs.  An  output  effectiveness  level  was  obtained  from  the  standards 
given  in  the  source  data. 

The  efficiency  analysis  using  only  the  number  of  sorties  as  an 
output  measure  is  summarized  in  Figure  5.  The  354th  Tactical  Fighter 
Wing  (TFW)  apparently  had  a  goal  of  approximately  480  sorties  per 
month.  This  thesis  used  480  as  the  standard  per  month;  however,  tKe 
reader  should  note  that  this  goal  could  change  with  time,  aircraft,  and 
wings.  The  CFA  analysis  using  only  one  output  changed  the  efficiency 
ratings  somewhat,  but  from  it  we  can  draw  some  basic  conclusions  that 
managers  and  analysts  can  use  in  all  CFA  analyses. 

The  first  thing  a  manager  will  note  after  a  basic  comparison  of 
efficiency  ratings  is  whether  the  unit  is  above  or  below  the  perfor¬ 
mance  goal.  Figure  5  can  be  thought  of  as  being  divided  into  four 
quadrants.  Quadrant  I  represents  units  that  are  efficient  as  well  as 
effective.  Clearly,  all  units  in  the  analysis  should  have  movement 
into  quadrant  I  as  a  goal.  Quadrant  II  represents  units  that  are 
efficient  but  fall  short  of  effectiveness.  Units  in  this  quadrant  will 
find  it  difficult  to  move  into  quadrant  I.  It  may  be  possible  to  force 


One  output  used  in  analysis 
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Figure  5.  AMU  Efficiency  vs.  Number  of  Sorties 


Che  unics  in  quadrant  II  Co  initially  become  inefficient  by  adding 
inputs  and  then  become  efficient  by  increasing  outputs  (18). 

Units  which  are  rated  inefficient  are  represented  by  quadrants  III 
and  IV.  The  three  circled  units  in  Figure  5  highlight  several  key 
points.  First,  two  units  can  have  the  same  efficiency  rating,  yet  have 
very  different  effectiveness  levels.  In  contrast,  two  units  can  have 
similar  effectiveness  levels  while  having  different  efficiency  ratings. 
The  ability  to  analyse  units  in  this  way  is  one  of  the  advantages 
of  the  CFA  model,  and  the  analysis  was  done  fairly  easily  from  the 
efficiency  ratings. 

When  the  sortie  standard  is  applied  to  Figure  3,  we  see  another 
useful  result  for  management.  Note  that  the  circled  unit  in  Figure  3 
represents  a  unit  that  is  efficient,  yet  its  effectiveness  in  producing 
sorties  is  lower  than  all  of  the  inefficient  units.  This  means  that 
this  unit  is  using  resources  in  the  best  way  possible,  but  they  are 
just  not  large  enough  to  achieve  greater  output.  For  this  unit  to 
increase  output,  it  will  be  necessary  to  increase  inputs.  In  her 
study  of  schools,  Linda  Reaves  (18)  suggested  that  a  strategem  for 
moving  units  from  the  efficient/ineffective  region  into  the  efficient/ 
effective  region  may  be  to  increase  the  inputs  for  these  units.  She 
felt  that  perhaps  this  would  force  the  units  to  become  temporarily 
inefficient  and  move  them  into  the  inefficient/ineffective  region. 

From  there,  she  proposed  these  units  could  reach  higher  output  levels 
and  eventually  move  into  the  efficient/effective  region.  This  type  of 
management  decision  hinges  upon  the  goal  of  organization.  If  output  is 
the  key  goal,  changes  should  be  made  in  the  input  mix.  However,  if 


efficiency  is  a  sufficient  goal,  the  unit  is  achieving  all  that  it 
needs  to  achieve. 

Figure  6  shows  a  similar  representation  for  analysis  using  mission 
capable  hours  as  the  comparison  output.  The  standard  for  this  analysis 
was  obtained  from  source  data  which  stated  the  goal  for  mission  capable 
hours  to  be  85  percent  of  total  hours.  Total  hours  represent  total 
hours  per  month  on  all  aircraft.  Thus,  a  standard  of  approximately 
16,642  hours  was  obtained  as  follows: 

(24  hours/day)(31  days/month) (26  aircraft/month/AMU)  * 

19,344  total  hours /month /AMU 

(19,344  total  hours/month/AMU)( .85  MC  hours/total  hours)  ■ 

16,442.4  MC  hours/month/AMU 

Movement  from  quadrants  III  and  IV  to  quadrant  I  can  be 
accomplished  in  two  ways  —  input  reduction  or  output  augmentation, 
respectively.  The  CFA  and  DEA  models  provide  the  manager  with 
information  aimed  at  making  these  kinds  of  decisions.  However, 
management  must  decide  what  level  of  performance  is  desired.  If 
effectiveness  is  the  sole  criterion,  an  effective  but  inefficient 
unit  may  be  acceptable.  If  we  presume  that  inefficiencies,  whether 
effective  or  not,  need  to  be  improved,  then  we  can  use  CFA  evaluation 
results  as  input  for  improvement  decisions. 

Phase  2:  Values  if  Efficient.  The  next  phase  of  analysis  is  to 
use  the  CFA  and  DEA  "values  if  efficient."  Using  the  efficiency 
ratings,  output  shortage  amounts,  and  input  surplus  amounts  from 
Table  III,  "values  if  efficient"  can  be  computed  for  inefficient  DMUs. 
It  is  important  to  note  that  all  outputs  and  inputs  must  be  adjusted 
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Figure  6.  AMU  Efficiency  vs.  Mission  Capable  Hours 
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for  the  unit  to  achieve  an  efficiency  rating.  Changing  only  a  subset 
of  the  inputs  or  outputs  will  not  result  in  an  efficient  rating. 

For  example,  the  "values  if  efficient"  can  be  computed  for 
Output  1  and  Input  2  of  DMD  7,  which  corresponds  to  the  353  AMD  in 
December  1983,  as  follows:  For  Output  1,  the  computed  shortage  (39.37) 
from  Table  III  is  added  to  the  observed  value  divided  by  the  efficiency 
measure  (423.0/. 99227)  which  yields  465.7  as  the  "value  if  efficient" 
for  Output  1,  shown  in  Table  IV.  A  "value  if  efficient"  for  Input  2 
is  calculated  by  subtracting  the  input  surplus  amount  for  Input  2 
(67.78)  from  the  observed  value  (843.2).  The  result  is  775.4,  which 
is  the  "value  if  efficient"  for  Input  2  as  shown  in  Table  IV.  These 
calculations  must  be  done  for  each  input  and  output.  This  gives  an 
input/output  mix  that  would  change  the  unit  from  an  inefficient  to 
an  efficient  rating. 

The  CFA/DEA  programs  provide  computed  "values  if  efficient" 
for  managerial  review  and  consideration.  This  feature  helps  make 
the  results  understandable  and  interpretable  without  requiring  that 
managers  be  familiar  with  the  mathematical  complexities  of  the  models. 

The  "values  if  efficient"  for  the  analysis  of  AMDs  using  both 
outputs  is  shown  in  Table  IV.  The  values  if  efficient  for  AMD  353  in 
December  83  will  be  used  to  show  how  managers  can  use  this  information 
to  improve  efficiency.  First,  the  models  determined  that  the  number  of 
sorties  should  increase  from  423  to  465.7.  This  may  not  be  possible 
because  of  weather  or  budget  constraints,  but  some  increase  may  be 
possible.  A  more  manageable  improvement  might  be  made  in  mission 
capable  hours  to  increase  them  from  16,481.3  to  16,611.3.  This  could 


be  accomplished  through  better  management  of  maintenance  personnel  and 
time,  improved  debriefing  and  troubleshooting,  or  faster  repairs. 

The  353  AMD  might  also  improve  efficiency  in  December  by  reducing 
three  of  its  inputs.  Inputs  1  and  3  are  not  decreased  by  the  model, 
but  that  does  not  restrict  managers  from  implementing  changes  in  these 
areas.  Input  1,  number  of  aircraft,  is  probably  beyond  the  control  of 
the  AMD  maintenance  officer  because  the  size  of  the  organization  is 
determined  at  higher  command  levels.  Input  3,  the  reciprocal  of  not 
mission  capable  supply,  is  another  input  managers  would  probably  not  be 
able  to  directly  affect  at  AMD  level.  The  model  shows  reductions  in 
the  remaining  three  inputs,  reciprocal  of  not  mission  capable 
maintenance,  number  of  flying  days,  and  fix  rate.  This  means  that  if 
the  observed  level  of  outputs  were  obtained  with  the  efficient  levels 
of  inputs,  the  unit  would  be  rated  efficient.  In  the  military, 
however,  it  would  be  more  advantageous  for  the  unit  to  try  to  increase 
outputs  rather  than  decrease  inputs. 

Phase  3:  Management  Ratios.  The  CFA/DEA  "values  if  efficient" 
can  also  be  converted  into  ratios  that  maintenance  managers  could  more 
readily  use  and  understand  (11:148).  For  example,  in  the  CFA  analysis 
of  353  AMD  in  December  83  (Table  IV),  a  sortie  rate  based  on  observed 
values  could  be  calculated  as  follows: 

423  sorties/month 

.  ■  17.3  sorties  per  month  per  aircraft 

24.4  aircraft 

Using  the  "value  if  efficient"  for  sorties,  a  new  efficient  sortie 


rate  can  be  computed: 


465.7  sorties/aonth 


19.1  sorties  per  month  per  aircraft 


24.4  aircraft 

A  aiailar  conversion  can  be  obtained  for  mission  capable  rate. 
Again  using  353  AMD,  December  83,  the  observed  mission  capable  rate  is: 

16481.3  Mission  Capable  hours/month 

-  -  21.8/24  -  .908 

24.4  aircraft  x  31  days/aonth  x  24  hours/day 

Similarly,  an  "if  efficient"  mission  capable  rate  can  be  obtained 
using  the  "value  if  efficient"  computed  by  CFA: 

1611.3  Mission  Capable  hours/aonth 

-  -  22/24  -  .916 

24.4  aircraft  x  31  days/aonth  x  24  hours/day 

Additional  ratio  conversions  are  possible.  They  are  useful 
because  maintenance  managers  are  accustomed  to  managing  with  ratio 
measures  and  output  rates. 

Comparison  of  Multiple  and  Single  Output  Analysis.  Three 
CFA  analyses  were  tried  for  the  AMD  data.  These  trials  included  an 
analysis  using  both  outputs  simultaneously,  followed  by  two  analyses, 
each  of  which  used  only  one  output.  A  comparison  of  the  efficiency 
ratings  is  shown  in  Table  V.  It  is  evident  that  the  selection  of  out- 
put(s)  can  dramatically  change  the  CFA  efficiency  ratings.  Inclusion 
of  both  outputs  gives  a  better  portrait  of  overall  efficiency. 

Correlation.  To  test  the  relationships  of  outputs  and  inputs,  a 
Pearson  correlation  analysis  was  performed  using  computer  programs  at 
the  University  of  Texas.  The  AMU  data  set  used  in  the  CFA  analyses  was 
used  for  the  correlation  analysis.  Results  of  the  correlation  are 
given  in  Table  VI. 


TABLE  V 


Comparison  of  Efficiency  Racings  for  AMD  Analysis 


SINGLE  OUTPUT  - 

SINGLE  OUTPUT  *  MISSION 


AMU* 

BOTH 

OUTPUTS 

SORTIES 

CAPABLE 

HOURS 

LB 

UB 

LB 

UB 

LB 

UB 

1 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

2 

1.00000 

1.00000 

.92051 

.92051 

1.00000 

1.00000 

3 

1.00000 

1.00000 

.86275 

.86275 

1.00000 

1.00000 

4 

1.00000 

1.00000 

1.00000 

1.00000 

.91096 

.96342 

5 

1.00000 

1.00000 

1.00000 

1.00000 

.89180 

.93395 

6 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

7 

.90146 

.99227 

.61079 

.64152 

.90146 

.99228 

8 

.95989 

.99292 

.64201 

.64201 

.95989 

.99293 

9 

1.00000 

1.00000 

.62041 

.86788 

1.00000 

1.00000 

10 

.93508 

.99194 

.73849 

.73849 

.93508 

.99059 

11 

1.00000 

1.00000 

.97518 

.97518 

.93083 

.98087 

12 

1.00000 

1.00000 

.88881 

.88881 

1.00000 

1.00000 

13 

.90002 

.95494 

.66723 

.66723 

.90002 

.95494 

14 

.81411 

.90167 

.70088 

.70088 

.81411 

.89874 

15 

.79550 

.91643 

.73049 

.73949 

.79550 

.91437 

*  See  Table  1  for  designation  of  AMU;  e.g.,  AMU  7  is  353  AMU, 
December  S3. 

UB  *  Upper  Bound 
LB  *  Lower  Bound 
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The  results  presented  in  Table  VI  shov  that  the  inputs  chosen 
were  not  significantly  correlated  with  each  other.  Highly  correlated 


inputs  can  be  viewed  as  measuring  the  same  thing  and  can  be  used 
interchangeably  without  affecting  CFA  results.  In  other  words,  if  two 
input  measures  had  been  highly  correlated,  one  could  have  been  omitted 
without  significantly  altering  the  efficiency  ratings  (12). 

Similarly,  the  output  measures  were  not  highly  correlated  with  one 
another.  However,  the  output  called  "sorties"  was  negatively 
correlated  to  the  reciprocal  of  not  mission  capable  maintenance,  flying 
days,  and  fix  rate  inputs.  The  flying  day  input  was  also  negatively 
correlated  to  output  two.  Apparently,  an  increase  in  the  number  of 
flying  days  does  not  correlate  with  increases  in  sortie  production  and 
mission  capable  time. 

Aircraft  Analysis 

General  Comments.  A  CFA  analysis  was  performed  on  the  simulated 
aircraft  data  discussed  in  Chapter  II.  The  efficiency  ratings  for  the 
aircraft  analysis  were  compiled  in  Table  VII.  The  aircraft  analysis 
would  have  had  more  meaning  if  a  large  empirical  data  base  had  been 
used.  However,  the  study  did  indicate  that  the  use  of  aircraft  as 
OMDs  was  feasible  and  worthy  of  follow-on  analysis. 

The  same  type  of  analysis  as  described  for  AMUs  was  performed  for 
aircraft  as  well.  Discussion  of  the  aircraft  results  will  be  brief  and 
will  highlight  a  few  interesting  points.  Referring  to  Figure  7,  one 
interesting  item  is  the  comparison  of  aircraft  10  and  11  which  have 
known  characteristics.  As  mentioned  in  Chapter  II,  these  units 


TABLE  VII 


CFA  Analysis  for  Aircraft  Efficiency  Ratings 


AIRCRAFT 

EFFICIENCY 

RATING 

AIRCRAFT 

EFFICIENCY 

RATING 

LB 

UB 

LB 

UB 

1 

1.00000 

1.00000 

15 

1.00000 

1.00000 

2 

1.00000 

1.00000 

16 

.67185 

.89178 

3 

.98433 

.99794 

17 

.87971 

.97681 

4 

1.00000 

1.00000 

18 

1.00000 

1.00000 

5 

1.00000 

1.00000 

19 

1.00000 

1.00000 

6 

1.00000 

1.00000 

20 

.95852 

.95852 

7 

1.00000 

1.00000 

21 

.96831 

.9764? 

8 

1.00000 

1.00000 

22 

1.00000 

1.00000 

9 

1.00000 

1.00000 

23 

1.00000 

1.00000 

10 

1.00000 

1.00000 

24 

1.00000 

1.00000 

11 

.97090 

.97090 

25 

1.00000 

1.00000 

12 

.94037 

.99232 

26 

1.00000 

1.00000 

13 

1.00000 

1.00000 

27 

1.00000 

1.00000 

14 

1.00000 

1.00000 

28 

1.00000 

1.00000 

UB  *  Upper  Bound 
LB  ■  Lower  Bound 
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40 


DMU  identified  by  number 
One  input  used  in  analysis 


CFA  EFFICIENCY  RATING 


Figure  7.  CFA  Efficiency  vs.  Number 
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I 


represent  hangar  queens.  Because  these  units  were  designated  as  hangar 
queens,  this  author  expected  them  both  to  receive  inefficient  ratings. 
However,  aircraft  10  was  so  unlike  the  other  aircraft  in  terms  of 
its  mix  of  inputs  and  outputs,  it  was  treated  as  an  "outlier"  and 
therefore  achieved  an  efficient  rating  by  ignoring  some  of  the  inputs. 
Specifically,  Inputs  2  and  3,  the  reciprocals  of  HMCM  and  NMCS 
respectively,  were  so  low  that  the  model  did  not  consider  these  inputs 
in  rating  aircraft  10.  In  this  case,  managers  would  want  to  perform 
further  analysis  of  aircraft  10  performance  which  might  require  the 
combined  use  of  judgment  and  the  CPA  and  DBA  analyses  in  order  to 
determine  whether  or  not  aircraft  10  is  truly  efficient. 

Aircraft  Performance  as  a  Measure  of  Unit  Performance.  An 
important  area  for  future  research  is  the  aggregation  of  the  individual 
aircraft  performance  ratings  into  a  measure  of  unit  performance.  This 
would  be  desirable  because  aircraft  measurements  provide  a  means  for 
objectively  evaluating  maintenance  performance  while  focusing  on  air¬ 
craft  performance  as  well.  There  are  several  possible  ways  to  combine 
individual  aircraft  ratings  into  a  unit  rating  that  will  be  discussed  in 
this  section,  including  visual,  ratio,  and  computer  program  assessments. 

Visual  Assessments.  An  easy  way  for  a  manager  to  get  a 
feel  for  unit  performance  from  analysis  of  individual  aircraft  is  by 
graphing  the  efficiency  rating  of  the  aircraft  and  assessing  this  graph 
visually.  An  example  of  this  type  of  analysis  is  shown  in  Figure  8. 

Note  that  aircraft  from  two  units  are  represented  and  distinguished 
from  one  another.  The  ratings  appear  to  be  grouped  by  unit.  A  manager 
might  conclude  that  unit  X  appears  to  have  better  overall  performance. 


1.0  - >  1.0 

CFA  EFFICIENCY  RATING 


Proposed  Visual  Unit  Comparison  Based  on  Aircraft  Evaluation 


The  visual  assessment  technique  is  a  starting  point  for  additional 


investigation.  If  results  do  not  group  into  distinct  areas  on  an 
efficiency  versus  output  graph,  additional  techniques  to  assess  unit 
performance  would  be  required. 

Ratio  Assessment.  After  the  visual  assessment  has  been 
performed,  a  ratio  assessment  could  also  be  performed.  For  example, 
units  could  be  compared  based  on  the  average  of  individual  ratings. 
If  aircraft  1  through  14  of  Table  VII  belong  to  unit  A  and  if  15 
through  28  belong  to  unit  B,  the  upper  bound  efficiency  ratings 
could  be  averaged  as  follows: 


£  aircraft  ratings 


number  of  aircraft 


Unit  A 
rating 


(ll  (1.0)  +  .99794  +  .97090  ♦  .99232) 
14 


Unit  B 
rating 


(l0  (1.0)  +  .89178  +  .97681  +  .95832  +  .97643) 
14 


.9972 


-  .98595 


Apparently,  unit  A  is  slightly  more  efficient  in  overall  performance 
than  unit  B. 

Another  ratio  that  could  be  developed  is  the  number  of  efficient 
aircraft  divided  by  the  total  aircraft  assigned.  Using  Figure  8  as 
a  reference,  we  can  compare  units  X  and  0  based  on  their  respective 
aircraft  performance  ratings: 


Unit  X 
rating 


9  efficient  aircraft  x  100 


14  total  aircraft/unit  X 


64. 3Z  efficient 


Unit  0 
rating 


6  efficient  aircraft  x  100 


42. 9Z  efficient 


14  total  aircraft/unit  X 

These  methods  of  analysis  give  the  manager  another  way  to  measure 
unit  performance. 

Computer  Analysis.  A  third  untried  method  for  aggregating 
individual  aircraft  ratings  into  a  measure  of  unit  performance  might  be 
to  use  the  individual  aircraft  ratings  as  inputs  or  outputs  in  a  unit 
CFA  analysis.  For  example,  the  percent  of  individual  aircraft  that 
were  rated  efficient  in  an  aircraft  CFA  analysis  could  be  viewed  as 
an  output  of  maintenance  in  a  CFA  analysis  of  the  unit. 

Summary 

In  summary,  this  researcher  noted  that  the  CFA  analysis  program 
gave  useful  information  for  maintenance  managers.  In  using  CFA  and 
DEA  it  must  be  stressed  that  the  programs  are  tools  and  not  absolute 
answers.  The  models  provide  empirical  evaluations  of  efficiency 
and  suggest  possible  sources  of  inefficiency;  however,  use  of  the 
efficiency  information  will  determine  the  value  of  the  models  for 
management. 

Figure  9  summarizes  the  phases  of  analysis  used  in  this  thesis. 
Note  that  CFA  analysis  could  be  part  of  an  iterative  performance 
evaluation  system.  This  thesis  is  an  example  of  one  analysis  iteration 
for  one  aircraft  maintenance  unit.  The  basic  analysis  plan  is 
applicable  to  all  types  of  CFA  analyses  for  performance  evaluation  and 
performance  improvement  programs. 


IV.  Cone luaions  and  Recommendations 

Introduction 

Chapter  IV  is  divided  into  two  sections.  The  first,  Conclusions 
examines  how  effectively  this  thesis  met  the  objectives  outlined  in 
Chapter  I.  The  second  section,  Suggestions  for  Future  Research, 
discusses  the  potential  uses  of  the  performance  evaluation  results 
in  this  thesis  and  identifies  areas  for  further  research  including 
the  need  for  CFA  application  to  other  areas  of  maintenance  management 

Conclusions 

This  section  is  organised  into  four  parts,  each  corresponding  to 
the  research  objectives  outlined  in  Chapter  I. 

Objective  One.  AMUs  and  aircraft  can  be  evaluated  using  the  CFA 
methodology.  In  this  researcher's  opinion,  CFA  exceeds  performance 
evaluation  techniques  currently  used  in  the  field  because  its  evalu¬ 
ations  are  empirically  based  and  because  CFA  provides  a  significant 
amount  of  information  which  can  be  analyzed  in  a  variety  of  ways. 

Objective  Two.  Several  methodologies  were  proposed  for  using 
individual  aircraft  analyses  to  measure  unit  performance.  Additional 
research  is  needed  before  firm  conclusions  can  be  reached  as  to  the 
feasibility  of  using  these  approaches.  Future  research  should  obtain 
empirical  aircraft  and  unit  data,  use  CFA  to  evaluate  both,  and 


Chen  determine  if  the  results  of  aggregating  individual  aircraft 
performances  correspond  to  the  aeasured  unit  performance. 

Study  of  individual  aircraft  performance  should  continue  to  be  a 
very  fruitful  area  for  research.  Not  only  could  this  type  of  analysis 
contribute  to  evaluation  of  aircraft  maintenance  units  but  it  could 
also  pinpoint  problem  aircraft  and  problems  in  supply  support. 

Objective  Three .  Constrained  Facet  Analysis  (CFA)  proved  to 
be  applicable  to  Air  Force  performance  evaluation.  CFA  could  be 
used  by  maintenance  managers  as  part  of  a  performance  evaluation  plan, 
but  managers  should  not  use  CFA  or  any  other  technique  as  the  sole 
basis  for  their  decisions.  Caution  must  also  be  used  to  avoid  over** 
emphasizing  efficiency  at  the  cost  of  effectiveness.  Further,  managers 
must  recall  that  CFA  computes  relative  rather  than  absolute  efficiency 
measures. 

CFA  would  be  difficult  for  wing  maintenance  managers  to  use 
without  training.  However,  continued  refinement  of  the  CFA  computer 
programs  would  make  it  easier  for  wing  level  managers  to  use  in  the 
future.  The  usefulness  of  CFA  would  be  enhanced  if  computer  systems 
were  available  to  enable  the  simultaneous  evaluation  of  several  wings. 
This  would  enable  managers  to  compare  several  units  and  would  spark 
healthy  competition  among  units  that  would  advance  the  efficiency 
frontier.  A  multi-unit  system  is  currently  being  used  in  the  Houston 
Independent  School  system,  and  Besaent  and  Bessent  (6)  report  that 
school  operations  in  Houston  have  improved  as  a  result  of  CFA. 

Objective  Four.  CFA  information  can  be  effectively  translated  for 
maintenance  management  use.  CFA  results  aid  in  operational  planning, 


•'.•'.•vi  rwv>v 


resource  allocation,  and  decision  asking,  end  they  can  also  be 
converted  to  ratio  aeasures  currently  used  by  managers  for  the  above 
functions. 

General  Conclusions.  In  summary,  Constrained  Facet  Analysis 
proved  to  be  effective  for  evaluating  aircraft  maintenance  units  and 
aircraft.  The  results  were  more  understandable  than  current  methods 
of  evaluating  performance.  CFA  also  provided  useful  suggestions  for 
iaq>rovement  which  managers  can  consider  when  choosing  among  alternative 
courses  of  action. 

Drawbacks  to  the  use  of  CFA  are  that  it  will  require  training  to 
implement  and  that,  at  this  writing,  the  software  is  available  only 
through  the  University  of  Texas  at  Austin.  However,  these  problems 
can  be  overcome. 


Suggestions  for  Future  Research 


Constrained  Facet  Analysis  could  have  a  widespread  impact  on 
maintenance  management  in  the  Air  Force.  Figure  10  is  a  matrix  of  the 
variety  of  management  levels  and  functions  which  could  benefit  from  CFA 
evaluations  of  units  and  aircraft.  The  four  major  levels  of  management 
are  wing,  major  command.  Air  Force  Logistics  Command  (AFLC),  and  Air 
Force.  At  each  level  CFA  results  could  be  used  to  support  strategic, 
tactical,  and  daily  operational  decision  making  in  areas  such  as 
resource  allocation,  operational  planning,  and  capability  assessment. 
Managers  at  all  levels  could  use  the  CFA  evaluations  as  a  source  of 
performance  information  which  could  help  them  plan  improvements  in 


their  units  and  in  the  entire  system.  CFA  might  also  be  used  for 
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special  applications  in  programs  such  as  the  Tactical  Air  Command  (TAC) 
dedicated  crew  chief  program.  CFA  evaluations  of  individual  aircraft 
might  be  used  to  help  evaluate  the  relative  performance  of  crew  chiefs. 

This  thesis  highlighted  the  need  for  additional  CFA  analysis  in 
the  aircraft  maintenance  field.  Future  research  is  needed  to  refine 
the  use  of  Constrained  Facet  Analysis  in  performing  maintenance 
evaluations.  These  research  needs  include: 

1.  Obtaining  more  extensive  data  bases  to  evaluate  units  over 
longer  periods  of  time  to  look  for  seasonal  affects  on  performance. 

2.  Obtaining  data  bases  on  a  variety  of  units  to  compare 
performance  in  different  locations. 

3.  Performing  research  to  identify  better  ways  to  select  input 
and  output  measures. 

4.  Continuing  to  study  the  performance  of  individual  aircraft  and 
how  individual  aircraft  performance  ratings  could  be  aggregated  into  a 
rating  of  unit  performance. 

Exploratory  research  to  broaden  the  application  of  CFA  to 
maintenance  is  also  needed  in  the  following  areas: 

1.  Determining  how  to  measure  the  support  provided  to  AMDs  by 
other  units  such  as  supply  or  off-equipment  maintenance  groups. 

2.  Comparing  the  performance  of  different  year  groups  and 
different  weapon  systems. 

3.  Determining  effects  of  extreme  environments  on  performance, 
e.g.,  comparing  how  cold  weather  performance  differs  from  performance 
in  more  temperate  climates. 


4.  Extending  the  use  of  CFA  to  studies  of  missile  and  vehicle 
maintenance . 

5.  Performing  CFA  evaluations  on  the  same  units  over  time  to 
detect  advances  of  the  efficiency  frontier  and  to  determine  whether 
improvements  in  performance  were  caused  by  CFA  feedback  and  managerial 
intervention. 

In  summary,  there  are  many  possible  uses  of  CFA  in  maintenance 
management.  CFA  would  give  managers  an  advanced  evaluation  technique 
upon  which  to  base  decisions  that  affect  performance. 


Appendix:  Acronyms  Used 


Air  Force  Base 

Air  Force  Institute  of  Technology 

Air  Force  Logistics  Command 

Air  Force  Systems  Command 

Aircraft  Maintenance  Unit 

Constrained  Facet  Analysis 
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